On-Line Recognition of Handwritten Mathematical Expressions Based on Stroke-Based Stochastic Context-Free Grammar

In this paper, we propose a new framework for online handwritten mathematical expression recognition. In this approach, we consider handwritten mathematical expressions as the output of stroke generation processes based on a stochastic context-free grammar which generates handwritten expressions stochastically. We estimate the most likely expression candidate derived from the grammar, rather than solving one by one the three major problems in mathematical expression recognition: symbol segmentation/recognition, 2D structure recognition, and expression syntax analysis. With this method, we can simultaneously recognize the symbols and structure of an expression within the grammatical constraint. Experiments revealed that this simultaneous estimation decreases errors in symbol segmentation and recognition, and that these errors are reduced as grammatical restriction is strengthened.